Determine image capture position information based on a quasi-periodic pattern

ABSTRACT

Examples disclosed herein relate to determining image capture position information based on a quasi-periodic pattern. For example, a processor may determine whether a target area is within a captured image based on the detection of a quasi-periodic pattern in a first detection area and in a second detection area of the captured image.

BACKGROUND

A mobile application may allow a user to capture an image, such as aphoto or video image. The image capture application may provide the userwith information about the image as it is being captured. For example, auser interface may display squares or other markings representing thefocus areas of the image.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings describe example embodiments. The following detaileddescription references the drawings, wherein:

FIG. 1 is a block diagram illustrating one example of a computing systemto determine image capture position information based on aquasi-periodic pattern

FIGS. 2A and 2B are diagrams, illustrating examples of capturing animage with a quasi-periodic pattern.

FIG. 3 is a block diagram illustrating one example of a method todetermine image capture position information based on a quasi-periodicpattern.

FIG. 4 is a flow chart illustrating one example of a method to processan image based on the presence of a quasi-periodic pattern.

FIG. 5 is a flow chart illustrating one example of a method to evaluatean area of an image within a detection area based on quasi-periodicpattern.

DETAILED DESCRIPTION

In one implementation, a processor analyzes a captured image to detect aquasi-periodic reference pattern to determine if the position of anobject in a captured image meets criteria for processing a targetportion of the image. For example, a printed or digital object may becreated with a quasi-periodic pattern as a border surrounding a targetportion of the object to corral or otherwise provide a reference for thelocation of the target portion. In one implementation, an image is takenof an object with a printed quasi-periodic pattern placed in a disposedlocation relative to the object. A processor may determine whether aquasi-periodic pattern is detected in particular detection areas of acaptured image to determine if the target area is positioned within thecaptured image. For example, the presence of the quasi-periodic patternmay be used to determine that the target area is not cropped or asufficient area of the target area is within the image. If aquasi-periodic pattern is detected in a captured image, informationabout the angle and scale of the pattern may be used to determine theorientation of the image capture, whether the image capture is toooblique, and whether the image capture distance is within an acceptablerange. In one implementation, a user interface is generated to provideuser feedback related to whether a captured image may be processedand/or how to improve the image capture to allow the image to beprocessed.

The quasi-periodic pattern border may be used to surround a target areaof the image such that the target area is between multiple portions ofthe border, such as where the quasi-periodic pattern border is used tocorral the target area. The border may be directly adjacent to thetarget area or provide some space between the target area and theborder. The border may confine the target area while not completelybordering or surrounding the target area. Using a quasi-periodic patternborder may allow for an aesthetically pleasing manner of providing imagecapture position information. For example, the border may be small ordecorative without drawing excessive attention to itself as an imagecapture guide. The border may be in multiple types of shapes dependingon the shape of the target area. A quasi-periodic border may allow formultiple acceptable capture positions, such as in a horizontal orvertical orientation and with multiple areas of the border that may bein a detection area. Detecting the quasi-periodic pattern in multipledetection areas may confirm that the target image is within the capturedimage without or with limited cropping and within acceptable orientationand resolution ranges.

The quasi-periodic pattern may be a halftone pattern created based onthe positioning of small black and white dots appearing as anaesthetically pleasing gray border. In one implementation, there is apayload associated with the quasi-periodic pattern such as with asteganographic halftone where the clustered halftone dots are shifteddots individually representative of payload data.

As an example, a user may capture a video image with a mobile device,and a user interface may be provide on the mobile device to highlighttwo detection areas. The user may attempt to place the two detectionareas in a manner that causes them to overlay the border pattern on thesubject of the video capture. A processor may determine whether thevideo image is suitable based on a detection of a quasi-periodic patternwithin the detection areas. The processor may further determineadditional aspects about the video capture based on the properties ofthe detected patterns. If the processor determines that the videocapture is sufficient, the processor may store the video frame. Thestored video frame may be used during processing of the target area ofthe image within the quasi-periodic pattern border. As another example,a high resolution image may be captured when the position criteria ismet.

FIG. 1 is block diagram illustrating one example of a computing system100 to determine image capture information based on a quasi-periodicpattern. For example, the computing system 100 may determine if aquasi-periodic pattern is present within a first and second detectionarea of an image. The presence of the quasi-periodic pattern in the twodetection areas may indicate that a target area is captured within theimage. For example, the quasi-periodic pattern may provide a referencepoint and/or border to a target area. Further analysis may be performedon the portions of the quasi-periodic pattern within and outside of thedetection areas to determine additional information about the imagecapture, such as orientation. The computing system 100 includes aprocessor 101 and a machine-readable storage medium 102. The computingsystem 100 may include a device to capture an image, such as a mobiledevice or other device capable of still image and/or video capture. Themobile device may transmit the image to the processor 101 via a network,such as the Internet.

The processor 101 may be a central processing unit (CPU), asemiconductor-based microprocessor, or any other device suitable forretrieval and execution of instructions. As an alternative or inaddition to fetching, decoding, and executing instructions, theprocessor 101 may include one or more integrated circuits (ICs) or otherelectronic circuits that comprise a plurality of electronic componentsfor performing the functionality described below. The functionalitydescribed below may be performed by multiple processors.

The processor 101 may communicate with the machine-readable storagemedium 102. The machine-readable storage medium 102 may be any suitablemachine readable medium, such as an electronic, magnetic, optical, orother physical storage device that stores executable instructions orother data (e.g., a hard disk drive, random access memory, flash memory,etc.). The machine-readable storage medium 102 may be, for example, acomputer readable non-transitory medium. The machine-readable storagemedium 102 may include image quasi-periodic pattern analysisinstructions 103 and image processing instructions 104.

The image quasi-periodic pattern analysis instructions 103 includesinstructions to detect and analyze a quasi-periodic pattern within animage. Detection areas may be positioned such that if the quasi-periodicpattern is within the detection areas, the target area is between thedetection areas within the image capture without or with acceptablecropping of the target area. There may be any suitable number ofdetection areas of any suitable shapes and sizes. In one implementation,there are multiple potential detection areas, and the processordetermines that the image capture is sufficient if a quasi-periodicpattern is detected within a subset of the detection areas. Thedetection areas may be positioned based on the size of the image suchthat they are in the same position regardless of the orientation orother capture differences. The portion of the image within a firstdetection area may be analyzed to determine whether it includes aquasi-periodic pattern. Any suitable method may be used to determinewhether a quasi-periodic pattern is present, such as a method ofanalyzing the image in the image domain or frequency domain. Forexample, a DFT may be created from the portion of the image within thedetection area. The processor 101 or another processor may analyze theDFT for peak frequencies. In some cases, a quasi-periodic pattern may bedetected where less than the entire detection area displays aquasi-periodic pattern, such as where 50% of the detection area includesa quasi-periodic pattern.

The image processing instructions 104 includes instructions to processthe captured image if the quasi-periodic pattern is detected within thetwo detection areas. The processor may store the image for processing ortransmit the image for processing. The image processing may involveperforming a service associated with the target area of the image. Forexample, processor may perform alignment methods to extract the targetarea of the image from the image based on the presence of thequasi-periodic patterns and properties of quasi-periodic pattern. In oneimplementation, the processor decodes a payload associated with thequasi-periodic pattern. The target area may be used as part of a mobileapplication, such as to provide information about an object in thetarget area of the image or provide a recommendation related to anobject in the target area of the image.

FIGS. 2A and 2B are diagrams illustrating one example of capturing animage with a quasi-periodic pattern. For example, FIG. 2A shows mobiledevice 200 captures an image of an object including a target 201 and aquasi-periodic pattern border 202 surrounding the target 201. The target201 may be a two-dimensional object including the pattern, such as apaper with a target portion and an attached surrounding frame, a hollowportion surrounded by a quasi-periodic frame that a user places over anobject of interest, or a combination of the two. There may be anysuitable number of objects and number of quasi-periodic borders, such aswhere there is an inner target and outer target separated by aquasi-periodic pattern.

The quasi-periodic pattern border 202 may surround the target 201 in anysuitable manner. For example, the quasi-periodic pattern border 202 maybe a continuous border, or may include gaps. In one implementation, thequasi-periodic pattern border 202 surrounds the target 201 on two sides,leaving other edges of the target 201 without the border. Thequasi-periodic border may be adjacent to the target 201 or includeadditional space. For example, where a farther capture distance isdesirable, there may be more space between the target 201 and thequasi-periodic pattern border 202. The mobile device 200 may capture aphotograph and/or video of the object.

FIG. 2B shows the display 203 of the mobile device 200. For example, thedisplay 203 may show an item as it is being captured in a video or stillimage. The display 203 shows a quasi-periodic pattern 204, correspondingto the quasi-periodic pattern border 202, and a target 205,corresponding to the target 201. The display 203 shows detection areas206 and 207 for detecting a quasi-periodic pattern. The detection areas206 and 207 may be shown on a user interface as displayed on the mobiledevice 200 overlaying the image. There may be any number of detectionareas in any suitable positions. The detection areas may be overlappingor contained within one another. The subset of detection areas may bedetermined based on detecting a quasi-periodic pattern in a firstdetection area, such as where a third detection area associated with aquasi-periodic pattern present in a first detection area, and a fourthdetection area is associated with a quasi-periodic pattern present in asecond detection area. In one implementation, a subset of the possibledetection areas are displayed on the user interface. For example, theremay be an inner and outer shape of detection areas in order to be ableto detect the border at different resolutions, and the inner or outershape of detection areas may be activated based on detecting a firstquasi-periodic pattern in a first detection area in either the inner orouter shape. In one implementation, the user interface displays thedetection area differently if a quasi-periodic pattern is detected inorder to provide user feedback as to which areas of the image should bemoved to be in a detection area. For example, detection areas with aquasi-periodic pattern detected may appear with a green outline, anddetection areas without a quasi-periodic pattern may appear with a redoutline.

FIG. 3 is a flow chart illustrating one example of a method to determineimage capture information based on a quasi-periodic pattern. Forexample, a processor may detect whether a quasi-periodic pattern ispresent within a detection area, and determine whether a captured imagemeets image capture criteria based on the detection. The quasi-periodicpattern may be used as a reference point for position information withinthe image, such as where the quasi-periodic pattern borders a targetarea. The method may be implemented, for example, by the computingsystem 100 of FIG. 1.

Beginning at 300, a processor determines whether a first portion of acaptured image within a first detection area includes a firstquasi-periodic pattern. The first detection area may be in any suitableposition. The detection area, may move dynamically or be in a staticposition. The detection area may be determined in any suitable manner.The detection area may be based on a position within a view finder. Theposition of the detection area may be based on the type of image beingregistered, such as based on the purpose or service associated with theimage capture. The detection area may be any suitable shape fordetermining whether the contents within the area includes aquasi-periodic pattern. There may be multiple detection areas such thata subset of a particular number or in particular positions detecting aquasi-periodic mark are used to determine that the image capture issufficient. In one implementation, the processor decides where to placethe detection areas and/or which detection areas to activate. Thenumber, shape, size, and position of the detection areas may be updated.

If determined that a first portion of a captured image within a firstdetection area does not include a first quasi-periodic pattern, theprocessor may provide an indication, such as by making a detection areaon a user interface appear a different color. The processor may notcontinue to process the target area if the first quasi-periodic patternis not detected.

The quasi-periodic pattern may be any suitable quasi-periodic pattern,such as a halftone pattern. The processor may detect the quasi-periodicpattern in any suitable manner. In one implementation, the frequency ofthe pattern may be detected based on peak frequencies within a DiscreteFrequency Transform (DFT) of the portion of the image within thedetection area.

If the first detection area includes a quasi-periodic pattern, theprocessor at 301 determines whether a second portion of a captured imagewithin a second detection area includes a second quasi-periodic pattern.The second detection area may be determined in any suitable manner. Inone implementation, the position of the second detection area may bedetermined based on the position of a first detection area. Thequasi-periodic pattern may be the same or different than the firstquasi-periodic pattern. The first and second quasi-periodic patterns maybe connected to one another and be part of the same pattern portion.

If determined that a second portion of a captured image within a seconddetection area does not include a second quasi-periodic pattern, theprocessor may provide an indication, such as by making the seconddetection area on a user interface appear a different color. Theprocessor may not continue to process the target area if the secondquasi-periodic pattern is not detected.

If the processor detects the first and second quasi-periodic pattern, at302 the processor determines based on the first quasi-periodic patternand the second quasi-periodic pattern whether the captured image meetscriteria to process a target area of the captured image. For example,the criteria may be met based on detecting the first and secondquasi-periodic patterns or based on the detection and analysis ofadditional characteristics. In one implementation, the processor furtherchecks additional properties of the first and second quasi-periodicpatterns to determine whether the image capture is sufficient. Forexample, the properties of the first and second quasi-periodic patternsmay be compared. For example, the peak frequency information in a DFTassociated with the first quasi-periodic pattern may be compared to thesecond quasi-periodic pattern to confirm that the difference is within athreshold range. In one implementation, the angle and/or scale of thefirst and second quasi-periodic patterns may be compared to determinethat the, patterns are associated with each other, the captureresolution, and/or the capture orientation.

At 303, the processor outputs information related to the determination.For example, if the criteria is met, the processor may store or transmitthe image for processing and/or process the target portion of the image.Processing may involve extracting the target portion of the image fromthe quasi-periodic pattern information.

FIG. 4 is a flow chart illustrating one example of a method to processan image based on the presence of a quasi-periodic pattern. For example,a user may capture still and/or video images, and a processor mayanalyze the images to determine if they sufficiently include a targetobject based on the presence of a quasi-periodic border. The processormay store the image for processing when it is determined that the targetobject is sufficiently within the captured image. In someimplementations, the processor may determine information about thecapture angle and resolution based on the quasi-periodic border, and thecapture angle and resolution information may be used to determinewhether to store the image for processing. The processor may provideadditional information to a user to indicate whether the image is readyfor processing and/or how to alter the image capture to improve theimage for processing. The method may be implemented, for example by thecomputing system 100.

At 400, a next image is received. For example, the image may be receivedfrom an image capture device associated with a mobile device, such as amobile phone. The image may be a video or still image. In oneimplementation, the user may continue to move the camera in relation toan object until a processor determines based on the presence of aquasi-periodic pattern that the captured image sufficiently captures thetarget area of the object. For example, each video frame may beconsidered to be the next image.

At 401-403, a processor associated with the device and/or a server forproviding a service to the device may analyze the contents of thereceived image in the detection areas 1-N to determine if they includequasi-periodic patterns. In one implementation, the analysis of adetection area depends on the results of the analysis from a previouslyanalyzed detection area. For example, a second detection area may not bechecked if a first detection area failed.

At 404, the processor determines if image position criteria is met. Thedetection areas may be evaluated together or one at a time. In oneimplementation, multiple detection areas are analyzed where theprocessor determines that the criteria is not met if a particular numberor set of detection areas do not include a quasi-periodic pattern. Inone implementation, more or different detection areas are analyzeddepending on which detection areas are determined to include aquasi-periodic pattern. In some cases, the processor may be able todetect the quasi-periodic pattern where the entire detection window doesnot include the pattern.

In one implementation, the processor analyzes criteria in addition tothe existence of the quasi-periodic patterns, such as propertiesassociated with the quasi-periodic patterns. For example, the scale andangle associated with the quasi-periodic pattern may be used todetermine the orientation of the image capture to determine if it is inan acceptable range. In one implementation, the processor comparesinformation about different portions of the quasi-periodic pattern. Forexample, the processor may determine that a quasi-periodic pattern isnot associated, with the border, such as where the scale is outside arange of difference as compared to other portions of the quasi-periodicpattern.

At 405, the processor outputs feedback based on the determination ofwhether the criteria is met. For example, the processor may cause a userinterface to be displayed that makes the detection areas visible, anddetection areas that met the criteria may be displayed differently, suchas with a different color outline, than those that did not meet thecriteria. The processor may determine feedback related to how to changethe camera capture. For example, an audio or visual cue may be providedto indicate that the image should be taken with the camera farther fromthe camera. The user interface may provide feedback related to the imagecapture, such as information about how to move the camera from side toside, to rotate it, or to move the camera closer or farther from theobject. If the criteria was not met, the processor evaluates the nextimage, such as the next camera photo capture or video frame.

At 406, the processor outputs image information if the criteria was met.For example, the analyzed image may be stored or a second image may becaptured and stored. The stored image may be stored on the devicecapturing the image and/or the device to process the image. In oneimplementation, a user captures a video image and moves the cameraaround until receiving positive feedback at which point the particularvideo frame is stored.

At 407, the processor processes the image information. Processing theimage may involve processing the image at different levels, such as theimage as a whole and/or the target portion. In one implementation,processing the image may involve an alignment and recovery method forseparating a quasi-periodic border from a target portion of the image.For example, alignment and recovery of the target area may be performedbased on information from the quasi-periodic pattern. The processor mayanalyze the quasi-periodic pattern in portions in addition to thedetection areas to align and recover the target area, such as the entirequasi-periodic pattern within the image. The processor may perform analignment and recovery method and output a quasi-periodic image and atarget portion image.

In one implementation, the processor performs further processing on thequasi-periodic image. For example, the processor may determine whetherthere is a data payload associated with the quasi-periodic pattern, andif so, may recover the data payload or transmit the image for payloadrecovery. The characteristics of the payload may then, in turn, indicateinformation related to further processing.

In one implementation, the processor performs further processing on thetarget image. For example, the target image may be rotated or otherwiseenhanced.

In one implementation, the target image is transmitted to anotherprocessor for providing a service related to the target image. Thequasi-periodic image may also be transmitted and/or the information fromthe data payload of the quasi-periodic image may also be transmitted.

FIG. 5 is a flow chart illustrating one example of a method to evaluatean area of an image within a detection area. For example, informationabout a border of a target area of an image may be evaluated todetermine if an image capture is ready for processing. Any suitableproperties of the quasi-periodic pattern may be used to determineinformation about the image capture. For example, the average number orposition of dots in a halftone image may be analyzed and/or compared toother portions of the image.

Beginning at 500, a processor determines whether the portion of an imagewithin a detection area includes a quasi-periodic pattern. For example,a Discrete Fourier Transform (DFT) of the image portion with thedetection area may be created and analyzed for peak frequency points. Ifthe portion includes a quasi-periodic pattern, further analysis isperformed.

Continuing to 501, a processor determines an angle associated with thequasi-periodic pattern. An angle associated with the quasi-periodicpattern may be used to determine an orientation of the captured image.For example, the pattern may be created with a particular orientationsuch that a 90 degree angle indicates a horizontal capture versus avertical capture. For a halftone image, the angle or orientation may bedetermined by comparing peak points in a DFT of the halftone image. Theangle may be used to determine if the image was taken from a horizontalor vertical angle so that the orientation of the target area may bedetermined.

Continuing to 502, the processor determines a scale associated with thequasi-periodic pattern. The scale of the pattern may be used todetermine information about the capture resolution. For example, a scaleoutside of a particular range, but indicate a capture resolution thatwould result in the image being unable to be processed. For a halftoneimage quasi-periodic pattern, the scale may be determined by the numberof samples per cell. An x and y scale may be determined and used toestimate the capture resolution.

Continuing to 503, the processor compares properties of quasi-periodicpatterns in multiple detection areas. For example, the scale ofquasi-periodic patterns in multiple detection areas may be compared,such as to determine whether the amount of difference is within aparticular range. A range outside of a threshold may indicate an obliquecamera angle or possible a pattern being registered as a quasi-periodicpattern that is not part of the quasi-periodic pattern border. In oneimplementation, the angle of multiple quasi-periodic patterns may becompared. Angles outside of a particular threshold range may indicatethat one of the quasi-periodic patterns is not associated with the samepattern and is not part of the quasi-periodic border. In oneimplementation, data payloads associated with quasi-periodic patternsmay be compared. Differences between the payloads and/or estimatednumbers of errors associated with the payloads may indicate that one ofthe quasi-periodic patterns is not associated with the same pattern andis not associated with the quasi-periodic border. Detecting andanalyzing quasi-periodic patterns within detection areas may result in amore aesthetically pleasing manner of providing image captureinformation and guidance.

1. A computing system, comprising: a processor to: determine whether atarget area is positioned within a captured image based on the detectionof a quasi-periodic pattern in a first detection area and in a seconddetection area of the captured image; and output information forprocessing the target area if determined that the target area ispositioned within the captured image.
 2. The computing system of claim1, wherein the computing system is further to provide feedback relatedto the image capture based on at least one of the presence or absence ofthe quasi-periodic pattern.
 3. The computing system of claim 1, whereinthe captured image is received from a mobile device and the storedcaptured image is transmitted to a computing device for providing aservice associated with the target area of the image.
 4. The computingsystem of claim 1, wherein the processor is further to determine apayload associated with the quasi-periodic pattern.
 5. The computingsystem of claim 1, wherein the processor is further to determine atleast one of the orientation and resolution of the captured image basedon the quasi-periodic pattern.
 6. A method, comprising: determining, bya processor, whether a first portion of a captured image positionedwithin a first detection area includes a first quasi-periodic pattern;determining whether a second portion of the captured image positionedwithin a second detection area includes a second quasi-periodic pattern;determine based on the first quasi-periodic pattern and the secondquasi-periodic pattern whether the captured image meets criteria toprocess a target area of the captured image; and output informationrelated to the determination.
 7. The method of claim 6, furthercomprising analyzing the first and second quasi-periodic patterns todetermine properties of the image capture of the image.
 8. The method ofclaim 7, wherein analyzing the first and second quasi periodic patternscomprise comparing a property of the first quasi-periodic pattern to aproperty of the second quasi-periodic pattern.
 9. The method of claim 7,wherein analyzing the first and second quasi-periodic patterns comprisesanalyzing at least one of the scale and orientation of the first andsecond quasi-periodic patterns.
 10. The method of claim 6, furthercomprising determining at least one of: the number of detection areas,the position of the detection areas, and the shape of the detectionareas.
 11. The method of claim 6, further comprising decoding a payloadassociated with the first and second quasi-periodic pattern.
 12. Acomputer readable non-transitory storage medium comprising instructionsexecutable by a processor to: cause a user interface to be displayedoverlaying an image being captured; detect if image contents positionedwithin a first detection area includes a first quasi-periodic patternand if the image contents positioned within a second detection areaincludes a second quasi-periodic pattern; output an indication on theuser interface related to the detection of at least one of the first andsecond quasi-periodic patterns; and output, information related to theimage for processing if the first and second quasi-periodic patterns aredetected.
 13. The machine-readable non-transitory storage medium ofclaim 12, further comprising instructions to provide feedback related tothe image capture based on the detection of the first and secondquasi-periodic patterns.
 14. The machine-readable non-transitory storagemedium of claim 13, further comprising instructions to provide feedbackbased on at least one of the angle and scale of the first quasi-periodicpattern.
 15. The machine-readable non-transitory storage medium of claim12, wherein instructions to output an indication comprise instructionsto create a visual indication such that a detection area including aquasi-periodic pattern appears differently than a detection area notincluding a quasi-periodic pattern.